The earliest video compositing devices to be put into service are generally known as "chroma-key" and were described by Kennedy and Gaskin of the National Broadcasting Company in the Dec. 1959 issue of the "Journal of the Society of Motion Picture & Television Engineers", pgs. 804 to 812.
The basic principle of all chroma-key devices is to develop a keying or switching signal based on the presence or absence of a backing color such as blue. The key signal is used to switch video from the background scene to the foreground scene when the foreground camera scanning beam leaves the blue backing and enters the subject area, and vice versa.
Recently, Nakamura et al modified the chroma-key system to a "soft edge chroma-key", in which the edges between the foreground and the background are purposely blurred. This tends to hide the hard edge effects of switching between the foreground and the background. However, detail is lost in the boundary area, and since individual strands of hair represent such a boundary area, they are not reproduced. Nakamura also added a subtraction circuit to eliminate the blue at the soft edge. The system is disclosed in the SMPTE Journal, Vol 90, No. 2, Feb. 1981, page 107.
Another recent modification of the chroma-key system is described by Mason of the British Broadcasting Corporation in UK patent No. 2,044,036, published Oct. 8, 1980. It also removes the blue tint in the soft edge of the switching region.
All chroma-key (i.e., switching devices) including the Nakamura and Mason soft edge devices suffer from a number of defects, among which are the loss of fine detail, such as hair, netting, lace, etc., and the inability to faithfully reproduce a full linear range of semi-transparent subjects.
The scanning beam in a switching system must be part way into the subject in order to develop sufficient information to determine that a switch action should occur. That portion of the subject covered by the scanning beam prior to switching is forever lost. No chroma-key device can therefore reproduce the foreground subject without loss of edge detail. This is why such devices cannot reproduce individual strands of hair.
A switching device cannot adequately reproduce a semi-transparent subject, since a switch is an "OR" device. That is, it shows either the foreground `or` the background scene through it. The visibility of the background scene should be reduced in luminance in proportion to the opacity of the foreground subject.
To accomplish this `and` combining of the foreground and background scenes it is necessary to keep the foreground video channels open at all times and at full level, so as to show all levels of the foreground subject transparency down to the smallest wisp of fog or clear glass. However, if the foreground channels are open the blue backing must be removed by suppression, not by switching. Further, the control of the background level must be linear; that is, proportional to the brightness and visibility of the colored backing. Proportional control of the background level not only results in realistic reproduction of transparent objects, it also causes retention and transfer of shadows from the backing to the background scene.
A non-switching compositing system, having the foreground channels always `on`, will retain the limiting resolution of the camera, and no detail will be lost.
The P. Vlahos U.S. Pat. No. 3,595,987 was filed Feb. 20, 1969. It introduced the concept of developing a control signal proportional to the brightness and visibility of the colored backing, of controlling the level of the background scene as a linear function of the amplitude of the control signal, and of eliminating the blue backing by limiting that video signal amplitude to a maximum that is represented by the instantaneous amplitude of one of the other primary colors. This limiting action also eliminated lens flare and discoloration of foreground subjects by secondary illumination from the colored backing when blue is limited to green. This patent issued July 27, 1971.
The technology of the patent requires a relatively pure backing color and does not disclose removal of the contaminating color when using a backing of low color purity; nor does it explicitly define the mixing as additive or non-additive.
The P. Vlahos U.S. Pat. No. 4,007,407, issued Feb. 8, 1977, introduces a blue clamp, in which blue (B) may exceed green (G) to the extent that green exceeds red (R); i.e., B=G+(G-R).sup.+. The term G-R permits reproduction of blue eyes and pastel blue colors while simultaneously eliminating lens flare and subject color contamination from secondary illumination from the backing. A color ratio is established to distinguish the colored backing from the subject when the backing illumination is non-uniform.
This patent also discloses the complete removal of R G B contaminating colors in the colored backing by subtraction of a bias voltage, and also the use of additive mixing of foreground plus background signals to form the composite. It discloses a background and bias control signal as E=B-G.
This patent was limited in that the system could not reproduce magenta colors, nor did it possess a capability of modifying shadow density.
The P. Vlahos U.S Pat. No. 4,100,569, issued July 11, 1978, introduced the third term in the blue limit equation; i.e., B=G+K.sub.1 (G-R).sup.+ +K.sub.2 (R-G).sup.+, which permits the full reproduction of magenta, thus removing all restrictions as to the range of hues reproduced. This patent also discloses a bias and control signal of the general form: EQU E.sub.c =K.sub.1 [B-K.sub.2 (K.sub.r R OR K.sub.g G)+(1-K.sub.2)(K.sub.r R AND K.sub.g G)-K.sub.3 (1-B)].sup.+
The significance of this equation is that the second term permits the reproduction and compositing of cyan and magenta in the same scene. The third term permits the use of colored backings such as cyan or yellow. The schematic circuit shows foreground and background signals to be combined by additive mixing by using an operational amplifier.
The P. Vlahos U.S. Pat. No. 4,344,085, issued Aug. 10, 1982, discloses a fourth term in the E.sub.c equation of the form -K(1-E.sub.c), which serves to prevent a partial turn-on of the background in black glossy areas that reflect the blue backing. It further discloses the use of a partially non-linear control voltage for the purpose of cleaning up (removing) unwanted footprints from a blue floor, and for removing unwanted detail, such as visible seams and joints in the backing. It also discloses a glare/noise circuit to reduce electrical "noise" in shadows, and to eliminate floor glare on back-lighted subjects. Also disclosed is an advanced shadow control that permits shadows to be lightened or darkened. It also introduced compositing against a black backing or a white backing.
The patent does not disclose the removal of color contamination by subtraction of a vector from the vector representing the R G B components in an encoded signal.
The P. Vlahos U.S. Pat. application Ser. No. 06/305,073, filed Sept. 24, 1981, now U.S. Pat. No. 4,409,611, issued Oct. 11, 1983, discloses the removal of backing colors by a subtraction process in which the R G B components of the backing are expressed by a generated vector, which vector is subtracted from the subcarrier representing the R G B components of the foreground scene. The subtraction vector is under the control of E.sub.b, which has the form of: EQU E.sub.b =k[(B-G).sup.+ -K.sub.1 (G-R).sup.+ -K.sub.2 (R-G).sup.+ ].sup.+
It also discloses an always open foreground channel and combining the foreground and background scenes by additive and non-additive mixing. The chroma of the backing as well as chroma contamination of the foreground subject is removed. Luminance of the backing is removed, but luminance cast upon the subject by the backing may be retained in whole or in part and may be controlled by the average luminance of the background scene.
Also disclosed is a modified background control signal of the form: EQU E.sub.c =K.sub.1 [B-K.sub.2 (K.sub.r R OR K.sub.g G)+(1-K.sub.2)(K.sub.r R AND K.sub.g G)-K.sub.3 (1-B)].sup.+
Matching chroma vector phase and amplitude and luminance level is accomplished by manual adjustments by the operator. Only the selection of the E.sub.b /E.sub.c ratio for control of chroma removal is disclosed as automated.